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Neurons in the visual cortex of all examined nnannmals exhibit orientation or direction 
tuning. New innaging techniques are allowing the circuit mechanisnns underlying 
orientation and direction selectivity to be studied with clarity that was not possible a 
decade ago. However, these new techniques bring new challenges: robust quantitative 
nneasurennents are needed to evaluate the findings from these studies, which can involve 
thousands of cells of varying response strength. Here we show that traditional measures 
of selectivity such as the orientation index (0/) and direction index (D/) are poorly 
suited for quantitative evaluation of orientation and direction tuning. We explore several 
alternative methods for quantifying tuning and for addressing a variety of questions that 
arise in studies on orientation- and direction-tuned cells and cell populations. We provide 
recommendations for which methods are best suited to which applications and we offer 
tips for avoiding potential pitfalls in applying these methods. Our goal is to supply a solid 
quantitative foundation for studies involving orientation and direction tuning. 

Keywords: Monte Carlo, sampling, neural data analysis 



INTRODUCTION 

In the visual cortex of all examined mammalian species (Hubel 
and Wiesel, 1959, 1962), many neurons respond strongly to bars 
or edges at a particular preferred orientation. In some mammals 
such as carnivores, primates, and tree shrews, these orientation- 
selective cells are organized into functional columns (Hubel 
and Wiesel, 1962, 1968; Humphrey and Norton, 1980), and in 
other animals such as rodents there are no maps of orientation 
selectivity yet individual cells exhibit strong orientation selectiv- 
ity (Girman et al., 1999; Ohki et al, 2005; Van Hooser et al, 
2005; Mrsic-Flogel et al., 2007). Further, a substantial subset of 
orientation-selective cells also exhibit direction selectivity (Hubel 
and Wiesel, 1962; Weliky et al, 1996). That is, they respond more 
strongly to a properly oriented bar moving in a preferred direc- 
tion as compared to any other direction. The functional organiza- 
tion and development of orientation- and direction-selective cells 
are the focus of intense current research. 

A number of measures have been devised to assess the strength 
and significance of orientation and direction selectivity for a 
given cell (Henry et al., 1974; De Valois et al, 1982; Swindale, 
1998; Ringach et al, 2002; Grabska-Barwinska et al., 2012). 
Traditionally, these techniques were applied to spike responses 
obtained from cells recorded extracellularly with microelectrodes. 
These cells were often identified as candidates for recording pre- 
cisely because they exhibited some substantial selectivity to an 
oriented test stimulus that was employed while the investigator 
moved the electrode, hunting for cells. 

New challenges — the advent of unbiased optical recording 
techniques such as 2 -photon calcium imaging that sample all cells 



regardless of selectivity (Stosiek et al., 2003; Kerr et al., 2005; 
Garaschuk et al., 2006) and the need to characterize cells in devel- 
oping animals with poor, emerging selectivity — have introduced 
new difficulties for assessing orientation and direction selectiv- 
ity. Some of the traditional measures of selectivity can give noisy 
or spuriously high values when applied to cells that don't exhibit 
at least moderate selectivity. Further, exciting new molecular and 
circuit techniques are permitting the testing of very precise cir- 
cuit hypotheses about the mechanisms underlying orientation 
and direction selectivity. Knowledge about statistical power — the 
number of cells or repetitions of a stimulus that are needed in 
order to observe a change in selectivity of a particular size — is 
critical for these studies. 

Here we characterize the robustness of several measures of 
orientation and direction selectivity on simulated responses. We 
provide a recommendation for analysis methods for the prin- 
ciple questions that investigators usually ask: (1) How much 
orientation or direction selectivity does a cell exhibit? (2) Does 
a cell exhibit significant orientation or direction selectivity? 
(3) Has a manipulation introduced a significant change in the 
amount of orientation or direction selectivity at the popula- 
tion level? Further, we provide tables for statistical power, to 
estimate the amount of data that would be required to accu- 
rately answer these questions. These methods could in princi- 
ple be extended to other sensory response properties or other 
modalities; however, their performance depends on the form 
of the underlying response function, so they may perform 
less reliably in domains aside from orientation and direction 
selectivity. 



Frontiers in Neural Circuits 



www.frontiersin.org 



August 2014 | Volume 8 1 Article 92 | 1 



Mazurek et al. 



Quantifying orientation and direction selectivity 



MATERIALS AND METHODS 
COORDINATE SYSTEMS 

There is no standard coordinate system for indicating orientation 
or direction space. In this paper, we use "compass" coordinates, in 
which a horizontal bar moving upward is considered to be mov- 
ing at 0°, and angles increase in a clockwise manner. Another 
common coordinate system is the Cartesian system, where 0° 
indicates a vertical bar moving to the right, and angles increase in 
a counterclockwise direction. One can transform between these 
two systems using the following equations: 

^cartesian — 90 ^compass 
^compass — 90 ^cartesian 

Because there is no standard coordinate system for orientation 
and direction, and because some readers may be unfamiliar with 
orientation and direction, it is helpful to use pictures to indi- 
cate stimulus orientation and direction in slides and in published 
figures, as we do here. 

ANGULAR ADDITION 

In several equations, we express angles in terms of the sum of 
angles. For example, for a direction tuning curve we define the 
positive orthogonal orientation as follows: 6onh+ = dpref + 90°. 
Note that these angles are summed modulo 360° in direction 
space and modulo 180° in orientation space. For example, in 
direction space, 359° +2° = 361° modulo 360° = 1°. 

RELATIONSHIP OF 01 AND Dl TO OTHER COMMONLY USED MEASURES 
OF ORIENTATION AND DIRECTION SELECTIVITY 

In this paper, we use 01 and DI as the normalized mea- 
sures of "peak to trough" orientation selectivity and direction 
selectivity (see "Results"): OI = {Rpref _on - Rorth) I Rpref _on and 
DI = {Rpref — Rnuii)/Rpref. Many papers use a slightly mod- 
ified version of these measures that we will call the OSI 

and DSI: OSI = {Rpref _ori - Rorth) I {Rpref _on + Rorth) and DSI = 

{Rpref - Rnidl) /{Rpref + Rnull)- Still Other papers use the "orthog- 
onal to peak" ratio to quantify orientation selectivity: 0/P = 
Rorth I Rpref _ori = 1 ~ OI. In making quantitative comparisons 
across papers, the reader should note carefully which is being 
used. 

The response at the preferred orientation Rpref _ori and the 
response at the preferred direction Rpref can be determined in 
different ways. In some measures, these are taken to be the best 
response to one of the stimulus orientations or directions that was 
explicitly measured; that is, if we measure responses at stimulus 
directions Oi, 62, . . . ,9„, then we choose the response at the best 
9i. In other measures, we perform a fit to the tuning curve, and 
choose the maximum value of the fit as Rpref _ori or Rpref- 

MONTE CARLO SIMULATIONS 

In Monte Carlo simulations of orientation and direction tuning 
curves, the "true" preferred angle and tuning widths were varied 
randomly so that a variety of tuning curve shapes were analyzed. 
Ranges were selected to correspond to typical values observed in 
VI neurons. Each underlying "true" curve was a double Gaussian. 
The underlying angle preferences were chosen according to a 



uniform distribution between 0° and 360°. Tuning widths were 
chosen randomly according to a Gamma distribution with shape 
3 and scale 6: a = {Gamma{5, 6) + 10°)/1.18. 

In many of the figures, we examined curves with 21 values of 
underlying OI or DI. These were produced in the double Gaussian 
equation (see Results) by setting the baseline rate C = 10 — (i — 
l)/2, Rp = {i - l)/2, Rn = {i - l)/4, for ; from 1 to 21. In other 
figures we examined 21 values of underlying DI: C = 0, Rp = 10, 
_R„ = 10 — {i — l)/2. Note that all of these curves with varying 
direction selectivity exhibit high orientation selectivity (that is, 
orientation selectivity and direction selectivity levels were not co- 
varied). 

To calculate statistical power for simulated 2 -condition exper- 
iments, we simulated underlying orientation or direction curves 
with exactly the OI or DI specified, and randomly varied the pre- 
ferred angle and tuning width as described above. We simulated 
100 populations of increasing sizes, and calculated the minimum 
size when X% of these simulations produced significant differ- 
ences by applying a f-test with confidence X%, for X = 95, 99, 
and 99.9. 

NOISE MODELS 

In each set of simulations, the simulated noise parameter is 
described. We used two types of noise. The most common type of 
noise, intended to capture the statistics of spikes recorded with an 
extracellular electrode, was a constant Gaussian noise value that 
was added to responses of all orientations on all trials. This con- 
stant value is often expressed as a percentage of the maximum 
response, which was usually 10 Hz. So, 20% noise means 2 Hz 
noise was added to individual trial measurements. 

A more recent technique for recording neural responses is 2- 
photon imaging with Oregon Green BAPTA-1 AM (OGB-IAM) 
(Stosiek et al, 2003; Ohki et al., 2005; Garaschuk et al, 2006). 
This method involves bulk loading of neural tissue with a cal- 
cium indicator bound to an AM-ester, leading to the uptake of the 
calcium indicator by all neurons within the loading region. This 
technique produces noise characteristics that differ from extra- 
cellular spike recordings. Specifically, because 2-photon calcium 
imaging records intracellular calcium concentration rather than 
membrane voltage, it tends to show a lower signal-to-noise ratio 
and a constant background signal. By examining the responses in 
previous experiments (Li et al., 2008), we modeled this noise as 
Gaussian noise with magnitude equal to a constant factor plus a 
component that depended on the response at each direction, such 
that noise = 20% + 10% * response magnitude. 

RESULTS 

Orientation selectivity has been observed in the visual cortex 
of every mammal that has been examined, including carnivores 
(Hubel and Wiesel, 1962), primates (Hubel and Wiesel, 1968), 
rodents, including murid (Girman et al, 1999; Niell and Stryker, 
2008), and sciurid rodents (Heimel et al, 2005; Van Hooser et al., 
2005), and marsupials (Rocha-Miranda et al., 1976; Ibbotson and 
Mark, 2003). 

Orientation selectivity is traditionally assessed by sweeping a 
bar or by drifting sinusoidal gratings across the cell's receptive 
field in different directions (Hubel and Wiesel, 1962; Movshon 
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FIGURE 1 I Illustration of the assessment of orientation and direction 
selectivity. (A) Left: Depiction of a bar stimulus moving at different 
orientations across the receptive field of an example cell. The cell's 
responses to each orientation are indicated at the right. The preferred 
orientation is 45°. During each presentation of the bar stimulus, the 
stimulus moves back and forth in two opposite directions. This cell 
responds more strongly to movement of the bar toward 45° than it does to 
the opposite direction (225°). (B) A graph of responses to the same cell to 
sinusoidal gratings drifting in several directions. The cell gives the largest 
response {Rpref} to 45° {Op„f), and a weaker response {R„uii) to the opposite 
direction 225° {0nuii)- The cell responds less strongly to stimulation at either 
of the two orthogonal orientations (6orth+ and The cell's response 

decreases as the direction of the stimulus deviates from 8prgf', the 
difference between dprgf and the angle that causes the response to drop to 
half (Rft/,) its maximum value is called the half width at half height (0h„,/,/,). 



et al, 1978, pp. 101-120), although it can also be assessed by flash- 
ing static bars at dilferent orientations (Palmer and Davis, 1981). 
The example cell in Figure lA exhibits a substantial response to 
bars that are rotated at a 45° angle from horizontal. The stimulus 
in Figure lA moves in two opposite directions. The response to 
upward and rightward motion (45° ) is stronger than the response 
to downward and leftward motion (225°), indicating that the cell 
is sensitive not only to the orientation of the stimulus but also its 
direction of motion. Note that every cell that is direction-selective 
by this criterion is also orientation-selective, but an orientation- 
selective cell is not necessarily direction-selective, because a cell 
could exhibit equal responses to the two opposite directions. 

The responses of this example cell are shown on a graph in 
Figure IB. Each response is presented as a firing rate: the num- 
ber of spikes evoked by each stimulus has been divided by the 
duration of the stimulus in seconds. We imagine that the exper- 
imenter has collected some responses to "blank" stimuli, where 
the screen remains blank for the same time duration as for each 
bar stimulus, and has subtracted these "blank" or "background" 
responses from each measurement so that we are examining the 
contribution of the oriented stimulus to the cell's firing rate and 
not ongoing background activity. This collection of responses to 
a set of different orientations or directions is called an "orien- 
tation tuning curve" or a "direction tuning curve," respectively. 
The major descriptive features of orientation and direction tuning 
curves are illustrated. The stimulus angle that evokes the maxi- 
mum response Rp,-i,f is called the preferred direction {Opygf), while 
the opposite direction is called the null direction (Spre/ + 180°). 

QUANTIFYING THE DEGREE OF ORIENTATION AND DIRECTION 
SELECTIVITY 

From the graphical tuning curve in Figure IB, it is easy to imagine 
two major notions of orientation selectivity. One is a comparison 
of the cell's response to the preferred orientations {Rpref + Rnull) 
compared to the responses {Rorth+ and Rgrth- ) at the orientations 
that are orthogonal {eorth+ = Opref + 90°, e„th- = Opref - 90°) 
to the preferred orientation. This method has been employed in 
numerous studies, and we refer to it here as the orientation index 
(OJ): 

OI = (Rpref + Rnull " (Rorth+ + Rorth-) )/ {Rpref + Rnull) 

Note that it is not necessary to stimulate with directional stim- 
uli in order to obtain a measure of orientation selectivity. 
Indeed, in many studies, the bars are drifted back and forth 
and the responses to each pair of opposite directions are aver- 
aged together. In this "orientation space," the angle of stimulation 
ranges from 0° to 180°. We can calculate the orientation selectiv- 
ity index in this case by using the preferred response {Rpref _ori) 
and the response Rorth at the orthogonal orientation (Soi-A = 

Opref + 90°) 

OI = (Rpref _ori ~ Rorth) / Rpref _ori- 

The OI can nominally vary from 0 (no selectivity) to 1 (perfect 
selectivity), although it can exceed 1 if the response to the orthog- 
onal orientation drops below the background firing rate, that is, 
when Rorth is negative. 



In direction space, a direction index can be defined similarly: 

DI = {Rpref - Rnull) /Rpref. 

Another major notion of orientation or direction selectivity is the 
sensitivity of the response to the preferred angle. One can imagine 
measuring the amount one needs to change the orientation (or 
direction) angle from the preferred for the response to drop by 
some amount, such as by half (Rhh, the response at half-height). 
The angle difference 9i,„hh indicates how far in orientation space 



Frontiers in Neural Circuits 



www.frontiersin.org 



August 2014 | Volume 8 | Article 92 | 3 



Mazurek et al. 



Quantifying orientation and direction selectivity 



one must adjust the angle from the optimal to obtain half of the 
response height. This type of selectivity has been referred to as a 
cell's orientation tuning width. Measuring the orientation tuning 
width requires either substantial sampling of responses to differ- 
ent angles, or performing a tuning curve fit, which we will turn to 
later. 

Owing to the mathematical simplicity of the 01, most studies 
over the years have employed the OI to assess orientation selec- 
tivity. By ear, it is easy to assess whether or not a cell exhibits 
perfect orientation selectivity with the OI (that is, when there are 
no responses to the orthogonal angle), and the 01 is very easy to 
compute numerically even when one is measuring spikes or firing 
rates. When applied to cells with substantial selectivity, it provides 
a simple and valid measure of orientation selectivity. Figure 2 
shows simulated responses from a model neuron (i) with a the- 
oretical or "true" 07 of 0.77; we have simulated 10 repeated trials 
of each direction; 5 Hz of random noise was added to each simu- 
lated trial, and the means are plotted. The empirical 01, measured 
by taking the average of the responses at the preferred orientation 
subtracting the responses at the orthogonal orientation, and nor- 
malizing, we obtain a value (0.83) very close to the "true" OI value 
ofO.77. 

However, the case of a model cell (ii) with no orientation selec- 
tivity is also shown in Figure 2. Again, we have simulated 10 trials 
with 5 Hz of added noise. If we blindly report the empirical mea- 
sure of OI, we obtain a value 0.58, a value much larger than the 
"true" value of 0. The reason is that we always choose the angle of 
the empirical maximum response to be "preferred," and, in this 
case, that angle was just the angle that had the largest response due 
to noise only (there was no orientation signal). By random chance 
in this example, the responses at the angles that correspond to the 
orthogonal orientations (represented by the squares) are both less 
than the responses to the preferred orientation (represented by 
the triangles), so the 01 is large. 



In single unit recording studies in adult animals, one often 
ignores cells with weak responses, but if one is conducting an 
imaging study of lOO's of neurons, or developmental research 
in animals with weakly responsive cells, it is highly likely that 
some neurons will exhibit weak orientation selectivity. If the OI 
is applied blindly, it is likely that many of these weakly selective 
neurons will have empirical OI values that are high, only due to 
noise. 

VECTOR SPACES FOR ORIENTATION AND DIRECTION 

We can improve the situation by plotting the responses to indi- 
vidual stimuli in a vector space. In Figure 3, we have replotted 
the responses of model cells (i,ii) and a new model cell (iii) in 
polar plots. Figure 3A shows the responses plotted in orientation 
space, where values for responses to the two opposite directions 
have been averaged, and angles vary from 0° and 180°. Figure 3B 
shows the responses plotted in direction space, where angles vary 
from 0° to 360°. The graphs also show, in gray, the vector that 
is the sum of all of the mean responses in vector space. In these 
examples, the length of the vector sum is more related to the 
amount of orientation selectivity as compared to the OI index. 

The normalized length of this vector in orientation space is 
computed as follows: 

Eitm)exp(2i6>fc) | 

where R{6k) is the response to angle Ok- In direction 
length is the following: 



space this 



I^dir ■ 



j:^R(ek)exp{iek) 



The normalized vector length is related to a classic quantity in 
circular statistics called the circular variance (Batschelet, 1981; 




FIGURE 2 I Calculation of the empirical orientation selectivity index (01). 

(i) Simulated responses (10 trials, 5 Hz per trial noise) to a model cell with an 
underlying direction tuning curve indicated in gray. Error bars indicate 
standard error around the mean of the simulated responses. To calculate OI, 
responses from the preferred orientation angle are averaged together 
(triangles) and the responses to the orthogonal angles (squares) are 



subtracted. This quantity is normalized by the response to the preferred 
orientation angle (triangles). There is good qualitative agreement between 
the empirical 01 and the "true" OI. (ii) Same, for a model cell that is not 
orientation selective. The empirical 01 is still very large due to the noise in 
the simulated responses, and is not qualitatively similar to the "true" 
underlying OI. 
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FIGURE 3 I Responses of model cells in polar coordinates on the 
complex plane. (A) Responses of model cells in orientation space. 
Response (in spikes per second) at each angle is indicated by the 
distance from the origin. Orientation angles vary from 0 (horizontal) to 
90° (vertical) and back to 0°/180° (horizontal). Gray arrow indicates the 
vector mean of the responses to individual orientations. The normalized 
length of the mean response vector is the quantity L„j, which is 
1 minus the circular variance il-CirVar). Model cells i and ii are the 



same as in Figure 2. (B) Responses of the same cells plotted in 
direction space. Direction of motion is indicated, response at each 
angle is indicated by the distance from the origin. Note that 0° and 
180° both correspond to horizontal stimulus orientations, but moving in 
upward and downward directions, respectively. Gray arrow indicates the 
vector mean of responses to individual directions. Note that cell iii is 
highly orientation selective for oblique bars but is poorly selective for 
stimulus direction. 



Ringachetal, 2002): 



1 — CirVar = Lg 



We use the abbreviation CirVar to differentiate the circular vari- 
ance from the classic statistical quantity called "coefficient of 
variation," which is often abbreviated as CV. We similarly define a 
quantity called l-DirCirVar: 

1 — DirCirVar = L^ir 

This definition differs (by a factor of 2) from the classic definition 
of circular variance in direction space (Batschelet, 1981), but we 
leave off the factor of 2 here so that a cell with maximal selectivity 
has a l-DirCirVar of 1 (rather than 2). 

The vector lengths in orientation space (l-CirVar) and in 
direction space (l-DirCirVar) for the model cells i-iii are shown 
in Figure 3. There are two important things to notice in compar- 
ison to the OI. First, model cell ii has a high (spurious) empirical 



OI value (Figure 2ii) but has a small l-CirVar value, indicating 
that l-CirVar is closer to the true selectivity of the cell, which 
is 0. Second, a cell can only have a l-CirVar value of 1 when 
it exhibits a response to 1 orientation and no other orienta- 
tions. Cells with high selectivity that would have OI values near 
1, such as model cells i and iii, generally have lower l-CirVar 
values, since cells typically respond to more than a single ori- 
entation; that is, the response tuning generally has some width. 
Thus, circular variance depends on both selectivity and tuning 
width. 

COMPARING 01 AND CIRCULAR VARIANCE (AND Dl AND DIRECTION 
CIRCULAR VARIANCE) 

When one records a neuron experimentally, one can only obtain a 
limited number of samples of the neuron's responses. One would 
like to use these sampled responses to make the best guess about 
the neuron's "true" properties, which cannot be examined directly 
but can only be inferred from experimental observations. Here 
we used Monte Carlo simulations to consider which index, OI or 
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circular variance, allows one to make the best guess about the true 
orientation selectivity. 

We created 21 model orientation tuning curves that ranged 
in "true" selectivity from 0 to 1 (Figure 4A). From each model, 
we simulated 100 tuning curves with 10 experimental trials each 
by adding 50% single trial noise; the exact angle preference and 



tuning width was chosen randomly (see Materials and Methods). 
We then calculated the OI and l-CirVar for each simulation 
(Figure 4). 

The percentile distribution of empirical 01 values for each 
"true" OI value is shown in Figure 4B. There is a wide range 
of empirical OI values; for example, when the "true" OI is 0, 
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FIGURE 4 I Comparison of 01 and circular variance measures for 
simulated data. We created 100 simulations of tuning curves for each of 21 
underlying "true" tuning curves, ranging from 0/ = 0 to O/ = 1 , some shown 
in (A). Each trial had 50% noise added. (B) Percentiles of the empirically 
determined 01 for the 100 simulations at each underlying "true" 01 value. 
Note that for cells with 0 true selectivity, the empirical 01 values range from 
slightly negative to almost 0.5. (C) Percentiles of the empirically determined 
VOrVar index for each of the underlying "true" 01 values. Note that when 
"true" 01 is low, the VCirVar is always low. The index VCirVar increases as 
"true" 01 increases but the range of values remains narrower than the 
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corresponding range of empirical 01 values in (B). (D) The inverse of (B); 
given we observed an empirical 0/ value of x, what is the range of possible 
"true" 01 values that produced x in our simulations? An empirical 01 of 0 
could have arisen from cells with "true" 01 values ranging from 0 to 0.5, and 
an empirical 01 of 0.5 could have arisen from cells with a "true" 01 ranging 
from about 0.1 to about 0.8. (E) The inverse of (C). A VCirVar of 0 could have 
arisen from a "true" 01 ranging from 0 to about 0.3, and 1-CirVar of 0.25 
could have arisen from a "true" 01 ranging from about 0.4 to 0.8. The range 
of possible underlying "true" 01 values is much narrower when l-CirVar is 
used as a readout as compared to OI. 
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empirical 07 values ranged from slightly negative to about 0.5. By 
contrast (Figure 4C), the distribution of circular variance values 
is much tighter; when "true" OI is 0, the l-CirVar value is always 
nearly 0. 

While the results in Figures 4B,C show the range of empir- 
ical OI and 1-CirVar values that one might expect for a given 
"true" OI, the most relevant relationship for experimentalists is 
the inverse relationship: given that one observes an empirical OI 
value of X or a l-CirVar value of Y, what are the likely "true" 
OI values that could underlie these empirical values? As shown 
in Figure 4D, knowing the empirical OI value tells one very little 
about the "true" OI value: for example, if the empirical 07 is 0, the 
"true" OI could be as high as 0.5. On the other hand (Figure 4E), 
the circular variance gives more information about the "true" OI: 
if we obtain an empirical l-CirVar of 0, the "true" 01 is likely to 
be less than 0.3; if we obtain an empirical l-CirVar of 0.25, the 
"true" OI is likely to be between 0.4 and 0.7. 

We performed similar simulations for direction selectivity, 
comparing the empirical DI with the empirical 1-DirCirVar 
(Figures). The difference in uncertainty about the "true" DI 
between the DI and l-DirCirVar is less pronounced than the 
difference in uncertainty between the OI and l-CirVar, but nev- 
ertheless the empirical 1-DirCirVar provides more information 
about the "true" DI than the empirical DI. 

The Monte Carlo simulation results presented in Figures 4, 5 
provide strong evidence that circular variance is a more robust 
and reliable indicator of the amount of orientation or direc- 
tion selectivity than the OI or DI. The circular variance works 
well when selectivity is strong or weak. We recommend the use 
of circular variance whenever quantification of the amount of 
orientation or direction selectivity is necessary. 

Experimentalists are also interested in knowing how many 
stimulus trials and stimulus angle steps should be presented to 
the animal in order to provide a quality estimate of the neuron's 
true orientation or direction selectivity. We performed Monte 
Carlo simulations where we systematically varied the single trial 
noise, number of stimulus trials, and the number of stimu- 
lus angles in order to understand how these factors influenced 
error in uncovering the "true" OI or DI (Figure 6). As expected, 
more trials and more angles were always better, but 45° angle 
steps and eight trials (or 22.5° steps with four trials) appear 
to be the minimum required for a quality assessment of direc- 
tion selectivity. Naturally, this result depends on the reliability 
of the neuron being studied; neurons with lower responsive- 
ness or higher noise will require more trials and/or stimulus 
angles. 

SIGNIFICANCE OF ORIENTATION AND DIRECTION TUNING 

When one suspects a cell is selective for stimulus orientation 
and/or direction, it is often important to verif)' this selectiv- 
ity statistically. We need tools that allow us to answer the 
question "is a cell's selectivity for orientation/direction sta- 
tistically significant?" In principle, one could simply measure 
a selectivity coefficient on each trial and perform statistics 
on this distribution of coefficients. However, the flaw in this 
analysis is in determining the nuU hypothesis with selectivity 
coefficients, and the flaw applies whether one uses OIIDI or 



l-CirVar/l-DirCirVar. Specifically, for all these coefficients, the 

expected value in the absence of selectivity is greater than zero 
because any variance across stimulus angles, whether stimulus- 
driven or random, always produces coefficient values greater 
than zero. Thus, one could not use this method to prove that 
a particular measured coefficient was not simply produced by 
noise. 

We have found that the best way to detect selectivity is to mea- 
sure the magnitude of orientation or direction vectors (Figure 7). 
For this test, we organize data into "trials," where a trial is one 
response at each stimulus orientation or direction. The orienta- 
tion or direction vector is calculated on each trial as the vector 
sum of responses on that trial measured in orientation or direc- 
tion space, respectively. The magnitude of the vector correlates 
with the degree of selectivity, and the expected magnitude is zero 
for zero selectivity. Hence we can perform statistics on the distri- 
bution of vector magnitudes against the null hypothesis HO: that 
the magnitude equals zero. 

For detecting orientation selectivity we use Hotelling's T^-test, 
which is a multivariate generalization of Student's T-test, to ask 
whether the 2-dimensional mean of orientation vectors is signif- 
icantly different from [0,0] (Figure 7C). Figure 8 shows that this 
test reliably detects orientation selectivity. In this figure, a model 
cell is simulated at different levels of underlying OI and differ- 
ent numbers of trials. The figure shows the number of trials that 
would be needed to detect different 01 levels vs. 07 = 0 at three 
levels of sensitivity (95, 99, and 99.9%). The test is specific for 
orientation selectivity, with p-values distributed uniformly when 
07 = 0. 

For detecting direction selectivity, it is possible in principle 
to apply Hotelling's T^-test to direction vectors. However, we 
have found that this method of testing for direction selectivity is 
quite insensitive because direction space is generally sampled too 
crudely to provide a reliable distribution of direction vectors. To 
address this problem, we developed a new test which we call the 
"direction dot product test." This test uses both orientation vec- 
tors and direction vectors to assess the direction selectivity of a 
cell (Figure 7F). 

In the direction dot product test, the first step is to obtain the 
orientation axis of the cell by calculating the angle of the average 
orientation vector. Next, we calculate the magnitude of the projec- 
tion of each direction vector onto the orientation axis (this is what 
we call the "direction dot product" for each direction vector). This 
gives us a 1 -dimensional distribution of direction dot product val- 
ues, one value for each direction vector. Finally, Student's T-test is 
performed on the distribution of direction dot products with HO: 
mean = 0. The test yields a p-value for whether the average mag- 
nitude of a distribution of direction vectors is significantly greater 
than zero. 

The direction dot product reliably detects direction selectivity. 
Figure 9 shows the direction dot product test applied to direction 
vectors from a simulated cell's response. The cell is simulated at 
different levels of underlying DI and different numbers of trials. 
The figure shows the number of trials that would be needed to 
detect different DI levels vs. D7 = 0 at three levels of sensitivity 
(95, 99, and 99.9%). The test is specific for direction selectivity, 
with p-values distributed uniformly when DI = 0. 
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FIGURE 5 I Comparison of Dl and direction circular variance measures 
for simulated data. We created 100 simulations of tuning curves for each of 
21 underlying "true" tuning curves, ranging from D/ = 0 to D/= 1, some 
curves shown in (A). Each trial had 50% noise added. (B) Percentiles of the 
empirically determined Dl for the 100 simulations at each underlying "true" 
Dl value. Note that for cells with 0 true selectivity, the empirical Dl values 
range from about 0 to about 0.5. (C) Percentiles of the empirically 
determined l-DirCirVar index for each of the underlying "true" Dl values. 
Note that when "true" Dl is low, the l-DirCirVar is always low. The index 
^-DirCirVar increases as "true" 01 increases but the range of values remains 
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narrower than the corresponding range of empirical Dl values in (B). (D) The 
inverse of (B); given we observed an empirical Dl value of x, what is the 
range of possible "true" Dl values that produced x in our simulations? An 
empirical Dl of 0 could have arisen from cells with "true" Dl values ranging 
from about 0 to 0.5, and an empirical Dl of 0.5 could have arisen from cells 
with a "true" Dl ranging from about 0.1 to 0.7 (E) The inverse of (C). A 
1-DirCirVar of 0 could have arisen from a "true" Dl ranging from 0 to about 
0.4, and ^-DirCirVar of 0.25 could have arisen from a "true" Dl ranging from 
about 0.1 to 0.7 The range of possible underlying "true" Dl values is 
narrower when 1-DirCirVar is used as a readout as compared to Dl. 



QUANTIFYING UNCERTAINTY AND DIFFERENCES IN ORIENTATION 
AND DIRECTION PREFERENCES 

Another objective that arises when one has a cell with selec- 
tivity for orientation/direction is to estimate the uncertainty of 
the measured selectivity parameters. Above we described tools 
for asking whether selectivity was significantly greater than zero. 
However, one might also like to obtain a measure of dispersion 



(e.g., standard deviation) or a confidence interval (e.g., stan- 
dard error) for selectivity parameters. In principle, one could 
simply obtain this from the distribution of OI/DUl-OriCirVarfl- 
DirCirVar values measured on repeated trials. However, these 
measures behave inconsistently as response properties vary, 
especially as selectivity approaches zero, so estimating the dis- 
tribution of parameters or confidence intervals from them is 
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FIGURE 6 I The dependence of error in identifying tlie true 01 on neural preferred direction. (B) Dependence of error on tine number of trials. More 



noise and stimulus sampling. On the Y axis of all plots is the average 
deviation between the "true" Ol (or Dl) and the "best guess" of Ol (or Dl) 
based on the empirical 1-CirVar (or 1-DirCirVar). (A) Dependence of error on 
single trial noise as a percentage of the maximum response rate to the 



trials offer modest improvements in average accuracy. (C) Dependence of 
error on number of angle steps. Additional angle steps offer a big improvement 
in estimating the amount of orientation or direction selectivity present. 
(D) Dependence of error for assorted numbers of trials and angle steps. 



not very meaningful. Fortunately, selectivity measures can be 
successfully employed to ask more specific statistical questions. 

One common question, especially in the era of 2-photon 
imaging where many cells are recorded simultaneously, is to ask 
whether one population of cells has different average selectivity 
than another population (or, equivalently, whether a population 
recorded at one point in time has different average selectivity 
than the same population recorded at another point in time). The 
approach is simple: Measure selectivity coefficients from each cell 
in the two populations, and perform a 2-sample T-test to measure 



whether selectivity differs between them. The test can be per- 
formed using OI/DI or l-OriCirVar/l-DirCirVar values. Table 1 
shows results from simulations asking how many cells would be 
required to detect differences in underlying Ol or Dl at different 
levels of confidence. The table shows that, as seen above, circular 
variance performs better than OI/DI. 

As an aside, one might wonder whether statistics on raw 
vectors could be used to answer this question. Since vector 
magnitudes correlate with selectivity, why not compare the vec- 
tors between the populations to see if selectivity has changed? 
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FIGURE 7 I Vector-based statistical tests with orientation (A-C) and 
direction (D-F) responses. This figure uses data from a model cell with 
strong tuning: Underlying 01=0.9, D/ = 0.5, noise = 20%. 16 directions 
(22.5° steps) were tested. For the orientation analysis, opposite directions at 
the same orientation were averaged together. (A) One trial from the model 
cell plotted in orientation response. Note that a "trial" is defined here as one 
measurement at each stimulus orientation. (B) The response from the trial 
shown in (A), plotted in polar coordinates. Black: The response obtained at 
individual orientations. Gray: The vector sum of the responses at individual 
orientations. This is the "orientation vector" on this trial. (C) Orientation 
vectors from seven trials from the model cell. Gray circles show the 
orientation vectors from the seven trials. The p-value above the graph gives 
the result of Hotelling's 7"^-test, which tests for whether the 2-dimensional 



mean of this distribution of orientation vectors is different from 10, 0). 
(D) One trial from the model cell plotted in direction space. Here a "trial" is 
defined as one measurement at each stimulus direction. (E) The response 
from the trial shown in (D), plotted in polar coordinates. Gray: The vector sum 
of the responses at individual directions. This is the "direction vector" on this 
trial. (F) Direction vectors from seven trials from the model cell. Gray circles 
show the direction vectors from the seven trials. The dashed line is 
orientation axis from this cell, obtained by measuring the angle of the 
average orientation vector. Black lines show the projection of the direction 
vectors onto the orientation axis (the "direction dot products"). Numbers 
give the magnitude of the direction dot product for each direction vector. The 
p-value above the graph gives the result of Student's T-test applied to the 
direction dot product values against HO: Mean = 0. 



The answer is that vector magnitudes, while they do correlate 
with selectivity, also correlate with tuning width and response 
magnitude (see Figure 3). If two populations differ in any of 
these response parameters, they will produce different vectors. 
Hence a test that looks for differences in vectors can give a 



positive result even if the populations are equally selective. Note 
that this effect isn't a problem when testing for the presence of 
selectivity, as we do in Figure 7, because here the null hypoth- 
esis is zero magnitude, which can only occur when selectivity 
is zero. Thus, statistics on raw vectors are suitable for detecting 
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FIGURE 8 I Sensitivity and specificity of Hotelling's T-squared test for 
detecting orientation selectivity. (A) Repeated simulations were performed 
with a single cell at different levels of underlying 01 and different numbers of 
trials. 16 angles (22.5° steps) were used; noise = 40% at all conditions. 
Sensitivity of Hotelling's T-squared test was measured at three levels of 
significance: 95, 99, and 99.9%. For example, the black line shows the 



number of trials needed such that one would detect an 01 difference of X 
with 95% confidence. (B) A cell was simulated with seven trials at underlying 
01 = 0. The simulation was repeated 200,000 times and each time a p-value 
was measured against HO: 01 = 0. The frequency of observed p-values was 
uniform between 0 and 1 , which is what would be expected for an unbiased 
test by repeated sampling of an unoriented cell. 
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FIGURE 9 I Sensitivity and specificity of the direction dot product 
test for detecting direction selectivity. (A) Repeated simulations were 
performed with a single cell at different levels of underlying Dl and 
different numbers of trials; Ol = 1 for all simulations. 16 angles (22.5° 
steps) were used; noise = 4 Hz at all conditions. Sensitivity of the 
direction dot product test was measured at three levels of significance: 



95, 99, and 99.9%. (B) A cell was simulated with 7 trials at underlying 
Dl = 0, 0/=1. The simulation was repeated 200,000 times and each 
time a p-value was measured against HO: Dl = 0. The frequency of 
observed p-values was uniform between 0 and 1, which is what would 
be expected for an unbiased test by repeated sampling of cells that 
are indifferent to direction. 



the presence of selectivity, but not for detecting differences in 
selectivity. 

Another common question is whether some specific response 
parameter differs between two cell populations. For example, 
one might wish to look for differences in preferred orientation 
between two populations. In this case, a vector-based test can be 
useful. Orientation vectors are affected by preferred orientation, 
so differences in preferred orientation lead to different distribu- 
tions of vectors from the cells. Hotelling's T^-test (specifically the 
2-sample version of the test, analogous to the 2-sample Student's 
T-test) can be used to detect such a difference. Figure 10 shows 
the sensitivity of the 2-sample Hotelling's T^-test in detecting 
differences in preferred orientation between two populations of 
cells. 

However, this test must be used with caution. Vectors are 
affected by all response parameters including preferred orien- 
tation, tuning width, and response magnitude, so a positive 
result simply means that one or more of these parameters differs 



between the two populations; it cannot prove that the difference 
is in preferred orientation or any other single parameter. The test 
may be useful as a broad screen to detect generalized differences 
in response parameters. But if a difference in a specific response 
parameter is sought, the best method is to perform statistics with 
iterative fitting, as described below. 

EXTRACTING PARAMETERS OF ORIENTATION AND DIRECTION TUNING 
CURVES WITH FITS 

In order to address the question of how well a given population 
of neurons encodes the orientation or direction of a stimulus, it is 
often important to know the precise parameters of a cell's tuning 
function such as its tuning angle or tuning width. Previous work 
using Monte Carlo simulations (Swindale, 1998) found that the 
best method for estimating tuning parameters from orientation 
or direction responses is to fit these responses with a Gaussian 
curve. In orientation space, we can fit the responses using a single 
Gaussian: 
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Table 1 | Minimum number of cells per condition that are needed to distinguish underlying orientation or direction selectivity index 
differences for two noise models. 
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The top three rows assume that the control group has an underlying 01 that is 0.5, and the experimental group has the Increment Indicated In each row. The bottom 
rows assume that the control group has an underlying Dl that is 0. 15, 0.3, or 0.5 as specified, and the experimental group has the increment Indicated In each row. 
The middle and right columns show the minimum cells that are needed to distinguish the control and experimental groups with a T-test at several confidence levels 
(0.95, 0.99, 0.999) If the experimenter Is calculating Ol (or Dl) values as a readout (middle columns), or If the experimenter Is calculating 1-ClrVar (or 1-DirClrVar) 
values (right columns). The number of cells indicated is the number of cells required per condition (control or experimental), so twice this number would be required 
for a total experiment. Two noise models were used. In the data points to the left of the slash, 40% noise was added to the responses. In the data points to the 
right of the slash, "2-photon OGB-IAM" noise was used; that Is, noise = 20% + 10 * response. Angle step size was 22. 5° . Note that many fewer cells are needed 
to evaluate changes in orientation and direction selectivity If 1-CirVar or 1-DlrCirVar Is used as a readout as compared to 01 or Dl. 
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FIGURE 10 I Sensitivity of the 2-sample version of Hotelling's 
T-squared test for detecting differences preferred orientation 
between different cell populations. Cells were simulated with 
seven trials each. We systematically varied the size of the cell 
populations and the size of the difference in preferred orientation. 



We measured the sensitivity for detecting the difference at three 
levels of confidence: 95, 99, and 99.9%. (A) Simulations performed 
using single-trial noise of 40% in all conditions. (B) Simulations 
performed using "2-photon OGB-IAM noise": noise = 20 % + 
(10% X expected response). 



R{e) = C + Rpe 2c2 

where C is a constant offset, Opref is the preferred orientation, 
Rp is the above-offset response to the preferred orientation, 
angori(x) = min (x, x — 180, x + 180), wraps angular difference 
values onto the interval 0° to 90°, and cr is a tuning width 
parameter. If we wish to only analyze the portion of the response 



above the offset, then the tuning width (half- width at half-height) 
is equal to ^log 4cr (half-width at half height) (Carandini and 
Ferster, 2000). 

In direction space, we can use a double Gaussian with the 
following equation: 

R(e) = C + Rpe +R^e 
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where C and Opref are defined as before, Rp is the 
above-offset response to the preferred direction, R„ is 
the above-offset response to the null direction, and 
ang(jir(x) = min (x, X — 360, X -|- 360), wraps angular differ- 
ence values onto the interval 0° to 180°, and ct is a tuning width 
parameter. Again, if we wish to only analyze the portion of the 
response above the offset, then the tuning width (half-width 
at half-height) is equal to ^log 4cr (half-width at half height) 
(Carandini and Ferster, 2000; Moore et al., 2005). 

Although Gaussian fits are the best method for determin- 
ing response parameters (Swindale, 1998), in practice there are 
several pitfalls to avoid. Several data analysis packages, such as 
Matlab (Math Works) offer the ability to fit functions to data, but 
blindly applying a least squares fit to the data using the above 
functions often leads to poor fits. Common errors are described 
in Figure 1 1 A. This problem is intractable in neural data because 



one never knows the "true" underlying response function, so it's 
impossible to say for certain that one fit is better than another. 
Hence, here we simulate responses with a known underlying 
response function, allowing us to evaluate the quality of our fits 
objectively. 

To prevent poor fitting, we use 2 ad-hoc procedures. First, we 
provide several constraints on the fit parameters. We constrain 
the width parameter cr to be at least as large as a/2, where a 
is the angle step used for stimulation; we force C to lie in the 
interval [-M,M], where M is the largest response to any stimulus; 
and Rp and R„ are constrained to lie in the interval [0, 3 M]. 
Second, we start the search using initial conditions that we expect 
will result in a good fit: 6pref = Om where M = R{9m)> Rp = 
_R„ = M, C = 0, and we explore several initial values for cr = 
{a/2, a, 40°, 60°, 90°}. We take the fit with the lowest least square 
error for all these initial values of a as the best fit of the data. 
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FIGURE 11 I Gaussian fits for assessing orientation and direction 
selectivity. (A) Common errors witli unconstrained fits (gray lines). Left: the 
unconstrained fit has gotten stuck in a local squared error minimum, using a 
tiny tuning width to fit 2 points very accurately. Middle: The unconstrained fit 
has used a peak response Rp that is much larger than any point actually 
present in the data, and a physiologically implausible negative weight for the 
null direction. Right: The unconstrained fit has found a reasonable fit, but the 
parameters do not make physical sense. The unconstrained fit posits a 



constant offset that is highly negative, with large responses to the preferred 
and null directions. All of these fitting error can be solved by constraining the 
fit parameters to values that make physical sense (solid lines, see text). 
(B) Mean errors in tuning width, preferred angle, and 0/for Monte Carlo 
simulations of cells with the underlying 0/s in Figure 4. Gray patch indicates 
25-75% interval (C) Mean errors in tuning width, preferred angle, and 01 for 
Monte Carlo simulations of cells with the underlying D/s in Figure 5. Gray 
patch indicates 25-75% interval. 
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Using this ad-hoc fitting method, we can explore how well we 
can identify the tuning width and preferred angle for the same 
model cells we explored in Figure 4. The performance of the fit 
improves as the underlying OI increases (Figure IIB), although 
the error in angle preference is relatively large when Ol is small. 
Because this error in angle preference is large when OI is small, 
we use another ad-hoc rule: we never report tuning widths or 
angle preferences from fits unless the data exhibits significant ori- 
entation selectivity by the Hotelling T^-test. The simulations of 
the model cells of varying direction selectivity in Figure 5 are fit 
with double Gaussian functions in Figure IIC. All of the tuning 



curves in Figures 5, 11B,C exhibit significant orientation selec- 
tivity, so the fit of tuning width and DI is excellent. As expected, 
when the underlying DI is smaller than about 0.25, the noise in 
the empirical responses obscures which of the two opposite direc- 
tions along the preferred orientation axis is the "true" preferred 
direction. 

The relationships between fit quality and noise and number 
of stimulus trials and stimulus angles are plotted in Figure 12. 
As with the circular variance index values, more angle steps are 
always better, but 22.5° step sizes provide relatively high quality 
fits when used with 6-8 trials. 
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FIGURE 12 I The dependence of errors in identifying tuning width, 
preference angle, and OI/DI on neural noise and stimulus sampling. On 

the Y axis of all plots is the median error between the "true" underlying 
quantity and the value provided by the fit. (A-C) are in orientation space, and 
(D-F) are in direction space. (A,D) Dependence of error on single trial noise 



as a percentage of the maximum response rate to the preferred direction. 
(B,E) Dependence of error on number of angle steps. Additional angle steps 
offer a modest improvement in estimating the fit parameters. 
(C,F) Dependence of error on the number of trials. More trials offer modest 
improvements in average accuracy. 
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FIGURE 13 I Estimating the distribution of preferred direction using 
bootstrap methods. (A) Data from a cell recorded with 2-photon calcium 
imaging using OGB-1 AM before and after extended exposure to a motion 
stimulus ("motion training"). Bars show ±1 standard error of the response. 
Solid lines show best fits with double Gaussian functions. The dashed line 
indicates mean response to a gray screen. (B) Distribution of preferred 
directions in bootstrap simulations of the cell shown in (A). The data from 
this cell was randomly resampled with replacement, creating a "simulated" 
cell, and this simulated data set was fit with double Gaussian functions. 
This procedure was repeated 100 times in each training condition, yielding 
the observed distributions of preferred direction. "Unc." lists the 
preference uncertainty, meaning the percentage of simulations whose 
preferred direction differed from the mean direction by more than 90°. 



QUANTIFYING UNCERTAINTY IN FIT PARAMETERS USING ITERATIVE 
FITS 

Finally, one useful outcome of iterative fitting is that it can be 
used to estimate uncertainty in fit parameters and to do statis- 
tics on these parameters. The simplest method for doing this uses 
the Hessian matrix, which measures the steepness of the error 
function near the local minimum where the fit algorithm set- 
tles. The matrix is obtained by sampling the error function near 
the local minimum and measuring the partial second derivative 
of this function for each parameter; standard Matlab optimiza- 
tion tools produce the Hessian matrix as an output parameter. 
Once obtained, the Hessian matrix can be transformed to obtain 
standard errors of fit parameters, and these can then be used to 
perform statistics (Press et al, 1992). 

Unfortunately, the Hessian method does not work for fitting 
orientation and direction curves. Since the Hessian matrix repre- 
sents the second partial derivatives of the error function, it can 
only be obtained when the error function is reasonably smooth. 
As described above, achieving adequate fits of orientation and 
direction data requires strict constraints on the fitting procedure. 
Because of these constraints, the error function is not smooth and 
thus a meaningful Hessian matrix generally cannot be obtained 
when fitting orientation and direction curves. 

Another method for using iterative fitting to quantify uncer- 
tainty in parameters is the bootstrap method. In this method, 
samples of data are repeatedly selected at random, with replace- 
ment, and fits are performed to each sample. The distribution of 
parameters obtained in these fits provides a reasonable estimate 
of the parameter distribution in the underlying population (Press 
et al., 1992), and hence this distribution can be used to calculate 
standard errors and to do statistics. 

In a previous study we employed the bootstrap method to esti- 
mate the distribution of preferred direction in individual cells 
from 2-photon recordings before and after extended exposure to a 
motion stimulus (Figure 13, modified from Li et al.). To obtain a 
distribution of preferred direction, we used the N trials obtained 
from a cell and created a "simulated" cell by randomly resam- 
pling these trials N times with replacement. The simulated data 
was then fit with a double Gaussian as described above. This pro- 
cedure was repeated 100 times and the preferred direction was 
obtained for each simulation, yielding a distribution of preferred 
direction values from this cell. 

One way we used this distribution was to detect significant 
direction selectivity. We quantified the "uncertainty" in direction 
preference, which is the percentage of simulations whose pre- 
ferred direction differed from the mean preferred direction by 
more than 90°. This uncertainty can vary between 0 and 50%, so 
we interpret (uncertainty x 2) as a p-value for significant direc- 
tion selectivity. We found that the sensitivity and specificity of 
this method for detecting direction selectivity is similar to what 
we obtain with the dot product direction test described above. 

There are several drawbacks to the bootstrap method. First, 
the method is very computationally intensive, with a standard 
test requiring several days of computer time. More importantly, 
results obtained from the bootstrap method depend on a variety 
of factors that are not related to the data. Specifically, the out- 
come of the test depends on the fitting algorithm employed, the 



initial value used in the fit, and the constraints placed on the fit. 
Researchers using the bootstrap method must take care to record 
and publicize details about their technique so that others may 
reproduce their findings. 

An alternative approach that generates confidence intervals 
is a Bayesian approach, such as that described in Cronin et al. 
(2010). The authors develop methods for estimating the entire 
probability distribution of each parameter value. 

DISCUSSION 

Orientation and direction tuning are probably the most inten- 
sively studied response properties in the cortex. Historically, 
these studies have focused on cells with strong selectivity as 
determined by simple comparisons between preferred and non- 
preferred responses; cells without such obvious selectivity were 
often declared, simply, "unselective." However, the advent of 
advanced techniques for recording and manipulating neurons 
requires us to investigate subtle differences between cells and to 
extend our analysis to cells with low selectivity. We need statistical 
tools that are suitable for addressing these subtle questions. 

Traditional measures for quantifying orientation and direction 
selectivity rely on assigning the stimulus evoking the strongest 
response as the "preferred" stimulus for the cell and assign the 
opposite stimulus as "non-preferred." The most commonly-used 
measures, OI and DI, compare the strongest stimulus to orthog- 
onal stimuli (for OI) or opposite-direction stimuli (for DI). Our 
analysis shows that these measures are generally unreliable, espe- 
cially for cells that have low selectivity or high noise (Figures 2, 4) . 
The key flaw with OIIDI and related measures is that preferred 
and non-preferred stimuli are always taken from sampled val- 
ues of orientation/direction; if the true preferred stimulus lies 
between sampled values (which is likely to be the case), it wiU be 
missed. 
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Table 2 | Recommended methods for answering several common scientific questions involving orientation and direction selectivity. 



Question 


Recommended method 


Quantifying tfie degree of orientation selectivity 


^-OriCirVar 


Quantifying tfie degree of direction selectivity 


VDirCirVar 


Testing for significance of orientation selectivity 


Hotelling's T^-test on orientation vectors 


Testing for significance of direction selectivity 


Direction dot product test on direction vectors 


Comparing orientation selectivity between two populations 


2-sample Student's T-Xes\ on 1 -OriCirVar values 


Comparing direction selectivity between two populations 


2-sample Student's T-test on ^-D'lrCirVar values 


Screening for any difference in response parameters (e.g., preferred 


2-sample Hotelling's 7^-test on orientation vectors 


orientation, tuning width, peak fieigfit) between two populations 




Extracting response parameters such as tuning angle or tuning width 


Fit data with Gaussian (for orientation data) or double Gaussian (for direction data) 


Quantifying the confidence/uncertainty of response parameters such as 


Bootstrap method: Resample data with replacement, then fit resampled data with 


tuning angle or tuning width 


Gaussian (for orientation data) or double Gaussian (for direction data) 



To obtain an accurate estimate of preferred and non-preferred 
stimuli, one must extrapolate between measured values. Vector- 
based methods effectively extrapolate measured responses by cal- 
culating the vector average of responses on each trial. Specifically, 
for quantifying selectivity, we recommend l-OriCirVar (for 
orientation) and l-DirCirVar (for direction). These measures 
demonstrate greater reliability than OI/DI (Figures 4, 5) and 
they are more sensitive than OI/DI for detecting differences in 
selectivity between two populations (Table 1). 

Vectors can also be used to assess whether a cell's selectivity 
is statistically significant. In this approach, we ask whether the 
2-dimensional mean of orientation or direction vectors is signif- 
icantly from zero. Specifically, Hotelling's T^-test on orientation 
vectors is reliable for detecting orientation selectivity (Figure 8) 
and the direction dot product test is reliable for detecting direc- 
tion selectivity (Figure 9). 

In some cases, we need to probe beyond selectivity to ask 
about specific response parameters such as tuning angle or tuning 
width. Vectors can be used to detect differences in these parame- 
ters between two populations (Figure 10); however, vector-based 
methods cannot identify which particular parameter or param- 
eters are responsible for the difference. To answer such precise 
questions, we recommend another method of extrapolation: 
fitting data with Gaussian curves (for orientation) or double 
Gaussian (for direction). Swindale (1998) showed that least 
squared fitting with these functions provided the best method 
for extracting response parameters from orientation and direc- 
tion data. This method provides accurate estimates of response 
parameters for cells with significant selectivity, provided that the 
fitting routine is appropriately constrained to avoid erroneous 
local minima (Figures 11, 12). 

Fitting also offers a tool for estimating the uncertainty of 
response parameters via the bootstrap method, where the data 
is randomly resampled multiple times with replacement and fits 
are performed to the resampled data. This method generates 
a distribution of values for each parameter which serves as an 
accurate estimate of the true distribution (Figure 12). Hence this 
method allows precise statistical questions to be asked about each 
response parameter underlying a cell's response. Note that alter- 
native methods for fitting data and estimating parameters have 
been used (e.g., Cronin et al., 2010); we have not compared these 
methods to those described here. 



Table 2 summarizes our recommendations for which method 
is best suited to a variety of quantitative questions regarding cells 
with orientation and direction tuning. Our goal is to provide 
tools for researchers to ask more refined statistical questions than 
have been possible using traditional measures such as OI/DI. As 
research advances into the precise mechanisms underlying ori- 
entation and direction tuning, robust quantitative methods will 
be required to distinguish competing theories. We hope the tools 
presented here will help accomplish this goal. 
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